Skip to content

Quiet validation PR comments#23537

Draft
nubtron wants to merge 11 commits intomasterfrom
dd/quiet-validation-comments
Draft

Quiet validation PR comments#23537
nubtron wants to merge 11 commits intomasterfrom
dd/quiet-validation-comments

Conversation

@nubtron
Copy link
Copy Markdown
Contributor

@nubtron nubtron commented Apr 30, 2026

What does this PR do?

Adds quieter validation PR reporting for ddev validate all by marking validation report comments with hidden status metadata, skipping duplicate success comments, and allowing PRs labeled ci/skip-validation-comments to suppress validation report comments entirely.

Motivation

Validation reports should remain useful when they require attention or show recovery, but repeated successful validation comments add noise to active PRs. A label-based suppression option also gives maintainers an explicit way to keep PR conversations quiet while validations continue to run.

Changes

  • Add hidden success/action-required markers to validation PR comments.
  • Detect previous successful validation comments by metadata and skip posting when the current run also succeeds.
  • Add ci/skip-validation-comments label detection from the pull request event payload.
  • Delete existing validation report comments and post nothing when comment suppression is enabled.
  • Keep workflow step summaries unchanged so each run still has local validation context.
  • Add tests for success/failure/success transitions, metadata formatting, and label-based suppression.

Testing/Validation

  • ruff check --fix ddev/src/ddev/cli/validate/all/__init__.py ddev/src/ddev/cli/validate/all/github.py ddev/src/ddev/cli/validate/all/orchestrator.py ddev/tests/cli/validate/all/test_command.py ddev/tests/cli/validate/all/test_github.py ddev/tests/cli/validate/all/test_orchestrator.py
  • ddev --no-interactive test ddev -- tests/cli/validate/all/test_github.py tests/cli/validate/all/test_orchestrator.py tests/cli/validate/all/test_command.py
  • Attempted ddev release changelog new fixed ddev -m "Allow PRs to suppress validation report comments with a label.", but it could not run in this detached worktree because origin/master is unavailable.

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Add the qa/skip-qa label if the PR doesn't need to be tested during QA.
  • If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

PR by Bits - View session in Datadog

Comment @DataDog to request changes

@datadog-datadog-prod-us1 @nubtron
Quiet duplicate validation comments
6c0bb94 
Co-authored-by: nubtron <49084+nubtron@users.noreply.github.com>
@datadog-official
Copy link
Copy Markdown
Contributor

datadog-official Bot commented Apr 30, 2026

View session in Datadog

Bits Dev status: ✅ Done

CI Auto-fix: Disabled | Enable

Comment @DataDog to request changes

@nubtron nubtron requested a review from a team as a code owner April 30, 2026 09:00
@datadog-official
Copy link
Copy Markdown
Contributor

datadog-official Bot commented Apr 30, 2026

I can only run on private repositories.

@nubtron nubtron marked this pull request as draft April 30, 2026 09:00
@dd-octo-sts
Copy link
Copy Markdown
Contributor

dd-octo-sts Bot commented Apr 30, 2026

⚠️ Recommendation: Add qa/skip-qa label

This PR does not modify any files shipped with the agent.

To help streamline the release process, please consider adding the qa/skip-qa label if these changes do not require QA testing.

@dd-octo-sts
Copy link
Copy Markdown
Contributor

dd-octo-sts Bot commented Apr 30, 2026

Validation Report

All 20 validations passed.

Show details
Validation Description Status
agent-reqs Verify check versions match the Agent requirements file ✅
ci Validate CI configuration and Codecov settings ✅
codeowners Validate every integration has a CODEOWNERS entry ✅
config Validate default configuration files against spec.yaml ✅
dep Verify dependency pins are consistent and Agent-compatible ✅
http Validate integrations use the HTTP wrapper correctly ✅
imports Validate check imports do not use deprecated modules ✅
integration-style Validate check code style conventions ✅
jmx-metrics Validate JMX metrics definition files and config ✅
labeler Validate PR labeler config matches integration directories ✅
legacy-signature Validate no integration uses the legacy Agent check signature ✅
license-headers Validate Python files have proper license headers ✅
licenses Validate third-party license attribution list ✅
metadata Validate metadata.csv metric definitions ✅
models Validate configuration data models match spec.yaml ✅
openmetrics Validate OpenMetrics integrations disable the metric limit ✅
package Validate Python package metadata and naming ✅
readmes Validate README files have required sections ✅
saved-views Validate saved view JSON file structure and fields ✅
version Validate version consistency between package and changelog ✅

View full run

@nubtron nubtron added the qa/skip-qa Automatically skip this PR for the next QA label Apr 30, 2026
chatgpt-codex-connector[bot]
chatgpt-codex-connector Bot reviewed Apr 30, 2026
View reviewed changes
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6c0bb94c38

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread ddev/src/ddev/cli/validate/all/orchestrator.py Outdated
Comment thread ddev/src/ddev/cli/validate/all/github.py
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 30, 2026 •
edited
Loading

Codecov Report

❌ Patch coverage is 97.10145% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.79%. Comparing base (94d61dd) to head (d7b93c6).

Additional details and impacted files
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@datadog-official
Copy link
Copy Markdown
Contributor

datadog-official Bot commented Apr 30, 2026 •
edited by datadog-datadog-prod-us1 Bot
Loading

Tests

✨ Fix all issues with BitsAI or with Cursor

⚠️ Warnings

🧪 1 Test failed

❄️ Known flaky: test_e2e_profile_dell_poweredge from test_e2e_core_vs_python.py   View in Datadog   (Fix with Cursor) 
[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 01-check-apikey.sh: executing... 
[cont-init.d] 01-check-apikey.sh: exited 0.
[cont-init.d] 50-ci.sh: executing... 
[cont-init.d] 50-ci.sh: exited 0.
[cont-init.d] 50-ecs-managed.sh: executing... 
...

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 97.10%
• Overall Coverage: 87.28% (+0.14%)

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: d7b93c6 | Docs | Datadog PR Page | Give us feedback!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@chatgpt-codex-connector chatgpt-codex-connector[bot] chatgpt-codex-connector[bot] left review comments

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

Bits AI ddev dev/testing dev/tooling qa/skip-qa Automatically skip this PR for the next QA team/agent-integrations

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@nubtron